feat(search): make the workspace index policy visible and correct - #2368

What this does

Vendors flashgrep v0.2.15, aligns BitFun with the daemon's new base-snapshot protocol, and closes the gaps that left the managed index either computing the wrong thing or impossible to interpret from the UI.

Every number quoted below is measured, not estimated; the methodology and raw runs live outside this PR.

Why the binary and the protocol have to move together

This is a breaking upstream change: RepoStatus.rebuild_recommended is gone, replaced by the base-delta fields; SearchParams.allow_scan_fallback and the BitFun-only QuerySpec.before_context / after_context are gone too.

BitFun declared rebuild_recommended as required, so dropping in the new binary alone makes every status-bearing response fail to deserialize — open_repo, search and glob all become unusable. Confirmed by an stdio probe against the v0.2.12 binary.

The "rebuild recommended" badge loses its data source and becomes "index catching up", driven by base_advance_target_head.

Line text: we were rendering line numbers as content

All four of the daemon's search modes return positions only (LineMatch { path, line_number }, no line text). BitFun rendered the line number as if it were the text, so Grep emitted path:73:line 73 and the desktop search preview showed "line 73".

Content search now goes through search/grouped_line_matches and hydrates text from disk:

• Local (workspace_search/line_hydration.rs): truncate to max_results before reading, open each file exactly once, do the reads in spawn_blocking.
• Remote (remote_ssh/workspace_search/remote_line_hydration.rs): the local "one open per file" shape cannot be carried over — on SSH that is one round trip per file, i.e. 250 serial round trips at the default head_limit=250. Instead the request ("which files, which lines of each") is written as a manifest and fed to a single awk pass. 250 matches measured at 4 commands or fewer, typically 1.
• Both paths share services-core::filesystem::content_preview, so rendering consistency comes from shared code rather than two parallel implementations.

Auto-index threshold: roughly 2000 indexable files

The inputs to that decision have asymmetric cost, so the count runs in two commands: git ls-files --cached takes 89 ms on chromium, while adding --others --exclude-standard takes 4250 ms. The tracked pass returns early once the threshold is met and never pays for the untracked walk. Command::output() waits for the process to exit, so an early break only helps if the commands are split.

Builds go through a queue with a cross-workspace disk budget.

The decision has to be visible

The daemon reports needs_index both while the policy is still evaluating a workspace and after it has deliberately declined, so BitFun could only write a sentence that was true either way. The user saw a state that never changed, with no way to tell "stuck" from "nothing should happen here".

The policy's decision now travels with the status (WorkspaceIndexStatus.auto_index), so a small workspace reads "No index needed — only 216 files, below the 2000 threshold; searching directly is faster."

One caveat: the count breaks as soon as the threshold is reached, so Eligible carries a lower bound, not a real count. Only the BelowThreshold branch has a true number — which is exactly the branch that needs to display one. The Eligible copy says "at least N".

Remote SSH has no BitFun-side auto-index policy at all (the remote daemon decides), so it reports None and the frontend falls back to the previous wording.

A non-Git workspace no longer shows a red indicator

flashgrep refuses to open a directory that is not a Git worktree with a HEAD commit. That is a property of the folder, not an index fault, so the raw daemon error is normalized into a stable BitFun-owned sentence, the indicator stays neutral gray, and content search silently falls back.

Grep's -A / -B / -C

The daemon has no context-line support, and the two fields BitFun was sending were its own invention which the daemon ignored — meaning -A / -B / -C were silently doing nothing. Requests asking for context lines now route to ripgrep. render_workspace_search_content_lines is kept for when the daemon gains support.

Verification

• cargo check --workspace clean
• cargo test -p bitfun-services-integrations --features workspace-search --lib — 27 passed
• cargo test -p bitfun-services-integrations --features workspace-search --test workspace_search_contracts -- --ignored — starts a real daemon, indexes a temporary Git repo, and asserts the results carry real line text rather than "line N" placeholders
• The remote hydration shell_tests feed the generated command to sh -c and run real awk. The same script is byte-identical on BWK awk 20200816 (macOS), GNU Awk 5.4.1 and mawk 1.3.4, covering CRLF, embedded tabs, multibyte text, empty files, out-of-range line numbers, missing files and long-line truncation
• tsc --noEmit, eslint, vitest run src/tools/file-explorer (12 passed)
• Locale key sets identical across the three locales (114 each)

Not included

• The performance investigation write-up and the architecture doc update are kept out of this PR deliberately.
• 17 files whose only change is rustfmt line rewrapping (relay-service/db.rs, terminal/transcript.rs, and others) are unrelated to this work and were left out.